Distributed Reinforcement Learning for Flexible and Efficient UAV Swarm Control

Over the past few years, the use of swarms of Unmanned Aerial Vehicles (UAVs) in monitoring and remote area surveillance applications has become widespread thanks to the price reduction and the increased capabilities of drones. The drones in the swarm need to cooperatively explore an unknown area, in order to identify and monitor interesting targets, while minimizing their movements. In this work, we propose a distributed Reinforcement Learning (RL) approach that scales to larger swarms without modifications. The proposed framework relies on the possibility for the UAVs to exchange some information through a communication channel, in order to achieve context-awareness and implicitly coordinate the swarm's actions. Our experiments show that the proposed method can yield effective strategies, which are robust to communication channel impairments, and that can easily deal with non-uniform distributions of targets and obstacles. Moreover, when agents are trained in a specific scenario, they can adapt to a new one with minimal additional training. We also show that our approach achieves better performance compared to a computationally intensive look-ahead heuristic.Comment: Preprint of the paper published in IEEE Transactions on Cognitive Communications and Networking ( Early Access

Distributed Resource Allocation for URLLC in IIoT Scenarios: A Multi-Armed Bandit Approach

This paper addresses the problem of enabling inter-machine Ultra-Reliable Low-Latency Communication (URLLC) in future 6G Industrial Internet of Things (IIoT) networks. As far as the Radio Access Network (RAN) is concerned, centralized pre-configured resource allocation requires scheduling grants to be disseminated to the User Equipments (UEs) before uplink transmissions, which is not efficient for URLLC, especially in case of flexible/unpredictable traffic. To alleviate this burden, we study a distributed, user-centric scheme based on machine learning in which UEs autonomously select their uplink radio resources without the need to wait for scheduling grants or preconfiguration of connections. Using simulation, we demonstrate that a Multi-Armed Bandit (MAB) approach represents a desirable solution to allocate resources with URLLC in mind in an IIoT environment, in case of both periodic and aperiodic traffic, even considering highly populated networks and aggressive traffic.Comment: 2022 IEEE Globecom Workshops (GC Wkshps): Future of Wireless Access and Sensing for Industrial IoT (FutureIIoT

Rate-Constrained Remote Contextual Bandits

We consider a rate-constrained contextual multiarmed bandit (RC-CMAB) problem, in which a group of agents are solving the same contextual multi-armed bandit (CMAB) problem. However, the contexts are observed by a remotely connected entity, i.e., the decision-maker, that updates the policy to maximize the returned rewards, and communicates the arms to be sampled by the agents to a controller over a rate-limited communications channel. This framework can be applied to personalized ad placement, whenever the content owner observes the website visitors, and hence has the context, but needs to transmit the ads to be shown to a controller that is in charge of placing the marketing content. Consequently, the rateconstrained CMAB (RC-CMAB) problem requires the study of lossy compression schemes for the policy to be employed whenever the constraint on the channel rate does not allow the uncompressed transmission of the decision-maker’s intentions. We characterize the fundamental information theoretic limits of this problem by letting the number of agents go to infinity, and study the regret that can be achieved, identifying the two distinct rate regions leading to linear and sub-linear regrets respectively. We then analyze the optimal compression scheme achievable in the limit with infinite agents, when using the forward and reverse KL divergence as distortion metric. Based on this, we also propose a practical coding scheme, and provide numerical results

Bike Sharing and Urban Mobility in a Post-Pandemic World

The Covid-19 pandemic has abruptly changed well- established mobility patterns, as the need for social distancing and lockdown orders have driven citizens to reduce their movements and avoid crowded mass transit. In this context, we look at the case of New York City's, bike sharing system, one of the largest in the world, to gain insights on the socio-economic variables behind urban mobility during a pandemic. We exploit several sources of Smart City data to analyze the relationship between bike sharing, public transport, and other modes of transportation, deriving interesting insights for future urban planning, both city- wide and at the neighborhood level. The New York City case study shows some of the most important trends during the lockdown, and the combination between mobility and socio-economic data can be used to understand the consequences of the pandemic on different communities, as well as the future directions of expansion and management of the bike sharing system and urban infrastructure

Semantic Communication of Learnable Concepts

We consider the problem of communicating a sequence of concepts, i.e., unknown and potentially stochastic maps, which can be observed only through examples, i.e., the mapping rules are unknown. The transmitter applies a learning algorithm to the available examples, and extracts knowledge from the data by optimizing a probability distribution over a set of models, i.e., known functions, which can better describe the observed data, and so potentially the underlying concepts. The transmitter then needs to communicate the learned models to a remote receiver through a rate-limited channel, to allow the receiver to decode the models that can describe the underlying sampled concepts as accurately as possible in their semantic space. After motivating our analysis, we propose the formal problem of communicating concepts, and provide its rate-distortion characterization, pointing out its connection with the concepts of empirical and strong coordination in a network. We also provide a bound for the distortion-rate function.Comment: This paper has been accepted for presentation at the 2023 IEEE International Symposium on Information Theor

Semantic Communication of Learnable Concepts

We consider the problem of communicating a sequence of concepts, i.e., unknown and potentially stochastic maps, which can be observed only through examples, i.e., the mapping rules are unknown. The transmitter applies a learning algorithm to the available examples, and extracts knowledge from the data by optimizing a probability distribution over a set of models, i.e., known functions, which can better describe the observed data, and so potentially the underlying concepts. The transmitter then needs to communicate the learned models to a remote receiver through a rate-limited channel, to allow the receiver to decode the models that can describe the underlying sampled concepts as accurately as possible in their semantic space. After motivating our analysis, we propose the formal problem of communicating concepts, and provide its rate-distortion characterization, pointing out its connection with the concepts of empirical and strong coordination in a network. We also provide a bound for the distortion-rate function

Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compression

The coordination of robotic swarms and the remote wireless control of industrial systems are among the major use cases for 5G and beyond systems: in these cases, the massive amounts of sensory information that needs to be shared over the wireless medium can overload even high-capacity connections. Consequently, solving the effective communication problem by optimizing the transmission strategy to discard irrelevant information can provide a significant advantage, but is often a very complex task. In this work, we consider a prototypal system in which an observer must communicate its sensory data to an actor controlling a task (e.g., a mobile robot in a factory). We then model it as a remote Partially Observable Markov Decision Process (POMDP), considering the effect of adopting semantic and effective communication-oriented solutions on the overall system performance. We split the communication problem by considering an ensemble Vector Quantized Variational Auto encoder (VQ-VAE) encoding, and train a Deep Reinforcement Learning (DRL) agent to dynamically adapt the quantization level, considering both the current state of the environment and the memory of past messages. We tested the proposed approach on the well-known CartPole reference control problem, obtaining a significant performance increase over traditional approaches

Enabling URLLC in 5G NR IIoT Networks: A Full-Stack End-to-End Analysis

This paper addresses the problem of enabling inter-machine ultra-reliable low-latency communication (URLLC) in 5th generation (5G) NR Industrial Internet of Things (IIoT) networks. In particular, we consider a common Standalone Non-Public Network (SNPN) architecture proposed by the 5G Alliance for Connected Industries and Automation (5G-ACIA), and formalize a full-stack end-to-end (E2E) latency analysis where semi-persistent uplink scheduling is considered in detail and compared with a baseline grant-based approach. Through simulations, we demonstrate that semi-persistent scheduling outperforms the baseline scheme and provides an E2E latency below 1 ms, thereby representing a desirable solution to allocate resources for URLLC. Notably, we provide numerical guidelines for dimensioning 3GPP-compliant IIoT networks for both periodic and aperiodic traffic applications, and as a function of the number of machines in the factory and of the offered traffic

Clinical parameters of COVID19 patients admitted to ICU.

Clinical parameters of COVID19 patients admitted to ICU.</p